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ABSTRACT 

Act Number 307, Public Acts of Michigan 1969, 
mandates that the Department of Education provide annual assessment 
of pupil achievement in basic skills — reading, vocabulary, English 
expression and mathematics,. The purpose of the assessment program is 
to provide information about groups of children for decision makers 
at the state level. A rationale for variable selection, the 
statistical methodology employed, and the procedural limitations are 
presented. The rationale emphasizes the strong relationship of pupil 
background (socioeconomic status, attitudes and aspirations) and 
school resource variables (human and financial) to each other and to 
student performance. The methodology emphasizes simple and easily 
comprehensible reporting, but includes multivariate analysis of the 
data; and program* s limits include problems of defining and measuring 
the complexities of an educational system. Measures of all variables 
were obtained in January 1970 for 310,000 pupils in grades four and 
seven. Analyses are in progress, (Author/ES) 
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The Department of Education shall provide for an 
annual test or tests of pupil achievements in the basic 
skills. Such test or tests shall provide for the objec- 
tive measurement of pupil learning outcomes in reading, 
mathematics, language arts and/or other general subject 
areas. Such test or tests shall be undertaken at one 
or more grade levels among elementary and/or secondary 
school pupils and shall be made statewide in application 
insofar as is necessary and possible.' 

When this piece of legislation was enacted and became law on August 
12, 1969, the Michigan Department of Education, Bureau of Research, hastily 
moved into action to implement this mandate so that a report could be 
assembled by Juna 15, 1970. Obvious questions are immediately evidentl 
What grade or grade levels should we assess? Should we sample students or 
items or both students and items? Should we measure the population? What 
variables should we attempt to index? Should we attempt the project in- 
house or contract with a testing firm? Are methodologies available and 
accessible to index the desired variables? Closure on these and similar 
issues was of top priority and closure came quickly. 

The Bureau staff after receiving positive feedback from the State 
Superintendent of Public Instruction and prior to introducing the program 
to the legislature devised several contingency plans ranging in magnitude 
from testing every child at three or four grade levels with a full length 
battery down to abolishing all plans for the program's implementation. 
Realizing the possible magnitude of the program, contact with five large 
testing firms was initiated for the purpose of securing information relative 
to the building and scoring of our desired test instrimient. The testing 
firms Immediately submitted information concerning their possibilities 
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and capabi 1 i ties wi th respect to the several phase* of our contingency plan. 
The legislature upon recommendations from the Bureau then authorized the 
spending of funds so thrt testing could be done at two grade levels with 
a shortened version of a full length test battery. The shortened test 
battery had the obvious advantages of financial savings and shortened 
testing time of pupils. As implied in the original act, each public school 
child was to be required to respond to the basic skills battery. Shortly 
after the authorization of funds, the Bureau enlisted the aid of Educational 
Testing Service to assist in developing and implementing the Basic Skills 
Program. 

The decision concerning which two grade levels to assess was made by 
the bureau staff. The fundamental consideration was that measures should 
be obtained at specific points in time in the students' academic progress. 
The consensus was that, ideally, one would like to have a measure of the 
child before he starts the educational process, probably preschool. This 
would be followed by a measure after the child's primary education of about 
grade three or four. This would be followed by a measure of post elementary 
school or grade six or seven. This would be followed by a measure of post 
junior high school orpgrade nine or ten. Finally, some measure would be 
taken of the total educational process or grade twelve. Methodological 
problems of unreliability quickly ruled out the preschool measure and 
dropout students complicated the grade twelve measure. Of the remaining 
grade levels, grades four and seven were selected as the areas of concen- 
t rat Ion. Because of the wording of the public act, a more precise defini- 
tion of basic skills was also required. Through research at the Bureau and 
in conjunction with Educational Testing Service, basic skilly were defined 



to be reading, English expression, vocabulary, and mathematics. 



The basic purpose of the assessment program is to provide information 
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for decision-makers at the state level. These decision-makers are concerned 
with groups of children rather than individual children. Therefore It was 
possible to adopt a shortened version of a full length test battery, since 
this would yield reliable and stable estimates of group parameters and be 
quicker and less expensive than the full measure. It would also not elimi- 
nate the need for testing programs at the local level. The savings in time 
and expense and consideration of local autonomy were all key political 
factors. 

Previous Research 

As previously stated, basic skills is not the only area of interest. 

The correlates of education were also considered to be very important. The 
work of Mort^, Flanagan 2 , Benson^, Burkhead, Fox and Holland**, Coleman** and 
many others all Indicate that there are prime considerations in education 
over and above the typical output measures. These variables must be con- 
sidered when one attempts to measure and interpret school performance. To 
look only at outputs of the system without regard for the inputs and 



Vaul R. Mort, "Studies in Educational Innovation from the Institute of 
Administrative Research: An Overview," IAR Research Bulletin, III (Oct. 

1 962 ) . 

2 

John C. Flanagan and others, A Survey and Follow-up of Educational Plans 
and Decisions in Relation to Aptitude Patterns': Studies of the American 

High School (Pittsburgh: University of Pittsburgh, 1962). 

3 

■'Charles I. Benson, State and Local Fiscal Relationships In Public Education 
in Cal ifornia (Sacramento: Senata of the State of California, 1965). — 

4 

Jesse Burkhead, Thomas G. Fox, and John W. Holland, Input and Output in 
Large-City High Schools (Syracuse: Syracuse University Press, 19©7). ” 

C 

■'James T. Coleman and others, Equality of Educational Opportunity (Washington, 
D. C.: U.S. Government Printing Office, 1966). 
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process of the system is meaningless. Therefore, the bureau staff, guided 
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by the previous literature and in consultation with Educational Testing 
Service, set out to index what they considered to be variables that are 
relevant and related to the educational system. After extensive review of 
the literature, numerous consultations with noted academic scholars and 
numerous bureau meetings, four areas were delineated as being highly related 
to academic achievement. These areas are socioeconomic status, the attitudes 
and aspirations of the students within a building, and the resources any 
given school can bring to bear upon its students. These school resources 
are considered to be both human and financial. Obviously there are other 
possibilities that merit consideration, but for the purposes of the first 
year of the program, these measures were chosen. 

Defining the Variables 

Exactly how do we define socioeconomic status, attitudes and aspirations 
school human resources and school financial resources? Realizing that we do 
not have a universally accepted definition of socioeconomic status and 
attitudes and aspirations or a rigorous method for indexing them, several 
arbitrary decisions were made. The socioeconomic status and attitudes and 
aspirations variables are defined in terms of the individual chi id* s response 
to the first section of the assessment battery. The first section of the 
battery is comprised of twenty-six items, thirteen of which attempt to index 
the child's attitudes and aspirations and twelve of which attempt to index 
socioeconomic status. These items are analogous to the items Coleman used 
in his study, Equality of Educational Opportunity and are indirect measures 
of the child's psychological self perceptions and home environment* An 
example of a socioeconomic status item might be, "Does your family have a 
dishwashing machine?" And an example of an attitude item might be, "If you 
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could change would you be someone different? 11 There were three viable 

■ j. 

choices of sources for obtaining socioeconomic status and attitude and 
aspiration data on a building level and these were the child, the teacher 
and the principal. In anticipation of a lack of reliable and valid data 
elicited from either school principals or teachers, the child was chosen 
as the source of this information. The school human and financial resource 
data are being collected from state records. This method of securing data 
has the advantage of time and cost savings. Unfortunately, however, this 
data is only available in district aggregate form. There is no possibility 
of getting these data for each building. Thus this aspe t of the program 
must be considered less than rigorous methodologically and falls prey to 
some of the criticism leveled at the Coleman data, i .e. , some of the greatest 
discrepancies between building resources is within districts rather than 
between districts. The primary variables under consideration for school 
human resources by district are average teacher salary, percentage of 
teachers with masters degrees or above, the average years of teaching 
experience per teacher, and the total pupil per professional personnel ratio. 
The variables under consideration for school financial resources by ui strict 
are local revenue per pupil, state school aid per pupil, elementary-secondary 
instructional expense per pupil, total current operating expenditure per 
pupil and state equalized valuation per pupil. It is realized that this 
listing is neither exhaustive nor indicative of all relevant school resources. 
However, considering the relevant literature on this subject In conjunction 
with the kinds of information that are available at the state level the 
selected variables all appear to be reasonable. Planning is currently under- 
way so that individual building data on school human and financial resources 
will be available in future years. 
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*,* The Indexing of the basic skills variables required a set of test speci- 
fications for each skill area. These spec! ficat ions were then reviewed and 
modified by teachers, administrators, curriculum consultants and bureau 
staff with respect to commonly used text books, teaching practices and 
recent literature in the State of Michigan. This reviewing procedure 
considered both relevancy and emphasis on material. Educational Testing 
Service then built the assessment battery according to these specifications. 

A proto-type of the final test battery was also subject to review and modi- 
fication. 

Administration of the Battery 

— L i: i i i i n — ■■■ 

During the last two weeks of January, 1970, approximately 310,000 
students in the public schools of Michigan were given the Basic Skills 
Assessment Battery. This massive effort would have been impossible without 
the cooperation of the local education agencies. A request to the local 
district superintendent by the Bureau of Research asked the superintendent 
to designate someone in his district to coordinate the testing program within 
the schools in the specific district. The support, cooperation and coordina- 
tion that was evidenced at this and, even more surprisingly, at the teacher 
level was overwhelmingly positive. This enthusiasm was much greater than 
we had anticipated and lends credence to the validity of the data. Hcwever , 
there ma^f be data missing from individual buildings and entire districts. 

The socioeconomic status and attitude and aspirations items became highly 
contested issues in certain communities with the outcome being that certain 
principals or district superintendents eliminated these items from the 
assessment battery. What effect this missing data will have upon the 

ultimate results remains to be seen. Missing data from individual students 
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will be taken into account during the consolidation and compilation of 
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building and district scores. 

Analyzing and Reporting the Data 

In any meaningful study, the analysis of the data must be guided by 
the purpose of and the specific questions asked by the study. A brief review 
of the basic questions may be beneficial at this time. The questions are: 

(1) what is the level of educational performance and its correlates in 

(a) Michigan and (b) in Michigan's geographic regions and community types? 

(2) Do districts (or schools) that score high (or low or average) on per- 
formance measures also score high (or low or average) on the correlates of 
education in (a) Michigan and (b) within Michigan's geographic regions and 
community types? (3) What are the scores of each of Michigan's districts 

on educational performance and its correlates? (*0 Do districts (or schools) 
that score high (or low or average) on socioeconomic status measures also 
score high (or low or average) on achievement and the correlates of education 
in (a) Michigan and (b) within Michigan's geographic regions and community 
types? (5) What is the present level of educational performance and its 
correlates within the State's large school district? (6) What changes over 
time may be noted in the answers to the above questions? 

All but the last of these questions, (i.e., what changes over time may 
be noted), can be addressed with the data from the 1969-70 assessment program. 
To answer the five basic questions a separate analysis will be conducted at 
each grade level. This analysis will attempt to relate the correlates of 
education to the outcomes of education in several specific ways. First the 
socioeconomic status and attitude and aspiration items will be factor ana- 
lyzed to attempt to reduce these variables to a workable number. Then the 
socioeconomic status, attitude and aspirations and basic skills variables 
for each student will be converted to standard scores. The zero order 
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correlation between all variables will be obtained. Because a shortened 
version of a full length test battery was adopted, the reliability of the 
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full length battery is necessarily reduced and because of this increase in 
unreliability, individual results will not be reported. The unreliability 
also has a tendency to underestimate the relationships between the variables, 
item analyses and reliability measures will be provided for the state as a 
whole and selected sub” population. 

Using the standard score for each pupil on each variable, two separate 
units of analysis will be obtained. One unit of analysis will be the school 
building score built by a composite of the students' standard scores within 
the building and the other unit of analysis will be the school district score 
built by a composite of the students 1 standard scores within the district. 

The district variables will be augmented by the school human and financial 
resources variables; these variables are not available at the building level. 
There will be a separate analysis for each grade level. There are approxi- 

A 

mately 3800 school buildings and 63** school districts in the State of Michigan 
* The state was divided up on the basis of geography arid community type 
and this led to four defined geographic regions, five defined community types, 
and the combinations of community type and geographic region. 

Definitions of Michigan's Community Types and Geographic Regions 
Community Types 

I Metropolitan Core ■ One or more adjacent cities with a population of 

50,000 or more which serves as the economic focal point of their environs. 

SI Cities * Communities of 10,000 to 50,000 population that serves as the 
economic focal point of its environs. 

III Towns * Community of 2,500 to 10,000 population that serves as the 
economic focal point of its environs. 

IV Urban Fringe * A community of any population size that has as its 
economic focal point a metropolitan core or a city. 

V Rural Community ■ A community of less than 2,500 population 




II Southern Michigan (excluding the Detroit Area) - All counties in 

southern Michigan that are south of and include Muskegon, Kent, Montcalm, 
Gratiot, Midland and Bay counties 



III Northern half of lower Peninsula ■ All counties that are north of region 
II and that are in the lower peninsula 

IV Upper Peninsula -This consists of all counties in the Upper Peninsula 



There is a slight methodological problem in the matrix of community type and 
geographic region — there are four empty cells (See Table I). For example, 
there are no metropolitan core cities in the upper peninsula of Michigan. 

Thus, certain statistical analyses are automatically eliminated on this aspect 
of these data. These regional and community type breakdowns will augment the 
analysis of the state as a whole and also allow for the comparisons of regions, 
community types, and the combinations of regions and community types with 
respect to the variables under consideration. 

There are many sophisticated analyses which could be undertaken, but 
these analyses are not readily interpretable or understandable to legislators,, 
state and local school officials, teachers and the lay public. Therefore, 
the initial analyses and reporting will be done to facilitate interpretation 
and understanding of the results at the level of the layman. This will be 
accomplished through several efforts at two distinct levels. These two levels 
are a public and a private reporting system. The private report will provide 
each local district superintendent with a score for each variable for his 
district and for each school building within his district. This report will 
speak to question number three. The public report will eliminate the names 
of buildings and districts and use these data to present additional infor- 
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mation. These building and district scores in isolation are relatively 
meaningless. Therefore, norm tables for the state as a whole, the four geo- 
graphic regions, the five community types and the sixteen combinations of 
regions and community types will be constructed showing the sample size, 
mean, standard deviation and the distribution of percentiles and frequencies 
for each variable. This will be done with each unit of analysis at each 
grade level resulting in a total of 104 norm tables. For example, the norm 
table for grade *» for the combination of Region I and Community Types II, 
using building scores as the units of analysis, may have 50 buildings on 
which to construct the norm table. This single norm table will contain a 
separate mean, standard deviation, sample size, percentile distribution and 
fr eq u e ncy distribution for each variable under consideration. The variables 
under consideration are socioeconomic status, attitudes and aspirations, 
reading, vocabulary, English expression and mathematics. It quickly becomes 
apparent that relatively few of the norm tables are appropriate for any 
given building. This will allow each district and building in the state to 
determine its relative status on each considered variable taking into 
account several independent variables, i.e., community type, geographic 
region and grade level. These reports will speak to question mmber one. 

In addition to these considerations, the socioeconomic status score 
will be broken into three categories, high, middle and low socioeconomic 
status, and will be used as a blocking variable to consider the relation- 
ships with the remaining correlates of education and the dependent variables, 
i.e., basic skills. This will be done for the state as a whole using both 
building and district scores as the units of analysis. Each of the geo- 
graphic region and community type breakdowns will use a socioeconomic status 
cutoff scores for the high, middle and low socioeconomic status categories 
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that are based upon only the buildings (or districts) that are contained 
within the specific region or community type. Thus, a region which is 
relatively affluent may have many buildings in the high socioeconomic status 
category for the state as a whole, but will have an equal number of buildings 
In the high, middle, or low socioeconomic status groups on the regional break- 
down. It Is, therefore, possible for a school to be classified in the high 
socioeconomic status category for the state as a whole and at the same time 
be classified in the middle or low socioeconomic status category on a 
regional or conrounity type classification. This conveys relevant and 
meaningful information if the mean and standard deviation In each category 
are reported. An additional consideration is that when one aggregates data 
at the district level according to the suggested breakdowns, the information 
contained in the data may be distorted, masked or both distorted and masked 
because of the wide discrepancies within districts. These reports speak 
to question number four. 

A third phase of the public reporting will be to obtain a total 
achievement score (the sum of the four basic skill standard scores) for 
each building and each district. Once again there will be two separate 
units of analysis resulting in two separate reports. The total achieve- 
ment score will be broken down into high, average and low achievement 
categories to be used as a blocking variable against the remaining variables. 
Categorization on region, co.7....'~.ii Cf type and the combination of region and 
community type will be analogous to the format specified in the socioeconomic 
status categorization. For example, the cut off scores used for categori- 
zation of the total achievement score will change from region to region and 
community type to community type. These reports will speak to question 
number two. 



Using the results of the data analysis, four of the five specific 
questions can be answered, l.e,, the status and distribution of basic skills 
education and their correlates in Michigan will be documented. Thus, the 
legislators will have an objective, methodologically rigorous body of infor- 
mation available to them upon which to base vitally important decisions con- 
cerning education. These analyses present descriptions of the relative status 
and distribution of the relevant variables and the relationships therein; 
these descriptions will be of the state as a whole, the four geographic 
regions, the five community types and the sixteen combinations of region and 
community type for both the fourth and seventh grades. 

Because of temporal constraints, these primary analyses must be sub- 
mitted to the legislature by mid-June; our target date ?s June 1, 1970. 
However, this does not eliminate the possibility and probability of addi- 
tional analysis. A multivariate analysis of variance will be employed 
using the categorized input variables as the independent variables and the 
basic skills variables as the dependent variables. Two separate multivariate 
analyses will be conducted; one analysis will use the school building as 
the unit of analysis and the other will use the school district as the unit 
of analysis. Other second order analyses call for various contingency tables 
and sub-samples of the population. Of particular interest is a comparison 
between school districts covering grades Kindergarten through twelve as 
opposed to those which do not span the entire thirteen grades. Another 
planned analysis deals with a multiple regression model being utilized to 
predict achievement from several input factors. It is not difficult to 
envision a number of additional analyses that may be of interest and 
importance. Certainly the bureau staff will not be able to exhaust these 
many possibilities due to time and resource constraints. Competent 





researchers outside the bureau may be interested in specific aspects of these 
data. 

In closing I would like to emphasize the fact that assessment has had 

a tremendous impact on the state of Michigan. A large Michigan school 

district writes In Its local assessment bulletin number seven: 

M ln all three areas, state, school district and individ- 
ual curriculum areas, assessment has forced us to take 
a new look at our goals. In this way the assessment 
program is proving valuable, even before we have any 
actual results from the assessment. 

The forth coming results are anxiously awaited and to say the least, Michigan 
legislators, state and local school officials, teachers and lay citizens 
are aware of the potential of the assessment program. With continued 
success, support and implementation of the assessment program, the ramifi- 
cations and possible consequences for reforming the educational enterprise 
in Michigan are considerable. 



^Lansing School System, Assessment Bulletin Member 7, February, 1970, 
Lansing, Michigan. 




